Does It Matter With What I Write

This study builds on three previous studies (Russell, 1999; Russell & Haney,1997; Russell & Plati, 2001) to examine the effect administering extended composi-tion test items on paper, on computer or on a portable writing device has on studentperformance. This study employs writing items from the 1999 Massachusetts Com-prehensive Assessment System (MCAS) to examine the mode of administration effectin grades four and eight. Similar to previous studies, this article finds that open-endedLanguage Arts items that require students to generate responses using paper andpencil severely underestimate the achievement of fourth grade students accustomedto writing using a computer. This study also finds that open-ended tests administeredon paper under-estimate the achievement of eigth grade students accustomed to writ-ing with an eMate (a portable writing device). Combining the effects found in thisstudy with those found in Russell’s 1999 study, this article estimates that the MCASLanguage Arts test under-estimates the performance of students accustomed to writ-ing using a computer by four to eight points on an eighty point scale. This article con-cludes by recommending that state testing programs that employ open-ended itemsin Language Arts provide students with the option of composing responses using thewriting tools with which they are accustomed to working.